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1. What is the purpose of this report? 

This report examines the outcomes of the Edgenuity application suites, which were widely adopted as an 
adjunct to virtual instruction during the shutdown of schools (fourth grading period of the 2019-20 school 
year) due to the Coronavirus pandemic. Edgenuity is a customizable standards-aligned video-based 
curriculum for middle and high school blended learning environments, which allows students to monitor 
their own progress while completing lessons, assessments, and interactive activities. The courses use 
audio, video, and text content along with different types of assignments including writing assignments 
and projects. Its curricular programs align with state standards and provide for secondary instruction and 
credit recovery in the form of either a standalone (credit bearing) or supplemental (non-credit bearing) 
basis. Students access the curriculum via the internet through their computer or tablet and begin the 
program with a pre-test followed by lecture-style lessons. 


2. Which populations are targeted in this report? 


The sample for the analyses of participation, usage, and outcome comprised students in Grade 6 through 
12 who during the 2019-20 school year were listed as enrolled in one or more core courses (i.e., English 
Language Arts, Mathematics, Science, and Social Science) during the spring semester, and used Edgenuity 
for one minute or more during the fourth grading period. The sample for analyses of impact was limited 
to students in Grades 6-8 with valid teacher-assigned core course-grades for the fourth grading period 
and who also had pre-test scores on the iReady diagnostic assessments (AP2, December 2019 
administration) in Reading and Mathematics. Students without complete data were excluded from the 
analyses. 


3. How were the data for this report collected and analyzed? 


Participation data were prepared by matching the M-DCPS student course registration data file to the 
Edgenuity cumulative usage file provided by the program vendor. Usage and participation were examined 
through descriptive statistics. Impact was analyzed by gauging whether variations in use were associated 
with variation in outcomes (as measured by fitting teacher-assigned grades to a continuous scale of 
measurement) once students' demographic characteristics and baseline achievement were considered. 
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4. How was Edgenuity deployed in the District? 
Edgenuity was used in 181 secondary school locations throughout the district as an adjunct to instruction 
in blended learning and in the form of standalone courses. Students were given the opportunity to use 


689 course modules in English Language Arts (n=102), Mathematics (n=284), Science (n=182), and Social 
Science (n=121). 


Schools. Table 1 lists the number of schools that implemented Edgenuity by grade organization and 
governance structure during the fourth quarter of the 2019-20 school year. 
Table 1. Edgenuity Deployment by School Type 


School Type 

Overall Traditional Charter 
Grade Span N % N % N % 
K-8 Centers 54 29.8 52 96.3 2 3.7 
Middle 52 =28.7 49 94.2 3 5.8 
Senior 55 30.3 47 85.5 8 14.5 
Other 20 11.0 18 90.0 2 10.0 
Total 181 100.0 166 91.7 15 8.3 


e 58.5% of the schools were middle or K-8 centers serving students in Grades 6 through 8 while 30.3% 
of the schools were senior high schools serving students in Grades 9 through 12. 
e 91.7% of the schools were traditional schools and 8.3% were charter schools. 


Format. Edgenuity was offered under two formats: as a blended learning tool (non-credit bearing) where 
it served as an adjunct to instruction with practice guided by a teacher, and as a standalone virtual course 
(credit bearing) where progress is guided by the application and the final grade is assigned by the software. 
It may be noted that a student using Edgenuity as a blended learning tool may be assigned to complete 
one, ten, or more units by the teacher, but the student is counted only once on Table 2, which follows. 
The table lists for each subject area and grade organization, the total number of student- Edgenuity- 
modules, followed by the number and percent of student-modules in each format (blended and 
standalone). 


Table 2. Edgenuity Student-Modules by Subject Area and Grade Organization, 2019-20 4 Quarter 


Middle Senior Total 

Blended Blended Blended 
Learning Standalone Learning Standalone Learning Standalone 
Subject Area N N % N % N N % N % N N % N % 
English Language Arts 28,722 28,514 99.3 208 0.7 16,533 16,154 97.7. 379 2.3 45,255 44,668 98.7 587 1.3 
Mathematics 28,610 28,332 99.0 278 1.0 11,397 10,995 96.5 402 3.5 40,007 39,327 98.3. 680 1.7 
Science 36,222 36,071 99.6 151 0.4 11,944 11,675 97.7 269 2.3 48,166 47,746 99.1 420 0.9 
Social Science 20,503 20,314 99.1 189 0.9 6,685 6,240 93.3 445 6.7 27,188 26,554 97.7 634 2.3 
Total? 51,702 51,328 99.3 374 0.7 33,886 32,813 96.8 1,073 3.2 85,588 84,141 98.3 1,447 1.7 


*Total represents an unduplicated count of students. 


e 99.3% of the middle level students used the blending learning format while 0.7% used the standalone 


format. 


e Overall, 96.8% of the senior level students used the blending learning format but use of the 
standalone format varied with grade from 2.3 to 6.7%. 

5. Which students used Edgenuity and how often did they use it? 

The remainder of this report is focused on the extent to which variations in the use of Edgenuity are 
associated with changes in students’ academic achievement. Due to the similarity of content across 
modules, student’s practice times on the modules within a given subject area are summed together to 
create an aggregated usage variable. Each student’s aggregate usage is then matched to all of their 
courses within that subject area, as recorded on the course registration file for spring 2020. The teacher- 
assigned grades serve as the outcome measures. Because the relationship between usage and outcome 
in standalone courses is fixed by the software, students with one or more such courses in a given subject 
area are excluded from further analyses of the outcomes of those subject areas. 


e Participation. Table 3 lists for each grade the total unduplicated number and percentage of students 
who registered for core courses during the spring of 2020, followed by the number and percentage of 
Edgenuity users enrolled in core courses, overall and by subject area. 


Table 3. Students Using Edgenuity by Subject Area 
Edgenuity Users 


Total English Language 
Spring Students Total Arts Mathematics Science Social Science 
Grade Enrolled N % N % N % N % N % 
Middle 
6 26,870 16,193 60.3 10,301 63.6 9,322 57.6 10,343 63.9 5,867 36.2 
7 26,406 17,049 64.6 9,177 53.8 9,347 54.8 11,057 64.9 8,444 49.5 
8 27,050 18,086 66.9 9,036 50.0 9,663 53.4 14,671 81.1 6,002 33.2 
Total 80,326 51,328 63.9 28,514 55.6 28,332 55.2 36,071 70.3 20,313 39.6 
Senior 
9 26,136 10,802 41.3 4,954 45.9 3,690 34.2 5,192 48.1 1,717 15.9 
10 25,532 9,406 36.8 4,674 49.7 3,790 40.3 4,406 46.8 108 1.1 
11 24,854 8,008 32.2 3,474 43.4 2,135 26.7 1,717 21.4 3,294 41.1 
12 24,344 4,597 18.9 3,051 66.4 1,370 29.8 357 7.8 1,118 24.3 
Total 100,866 32,813 32.5 16,153 49.2 10,985 33.5 11,672 35.6 6,237 19.0 


O Overall, 63.9% of all enrolled middle-grade students used Edgenuity modules at a teacher’s 
direction during the fourth grading period of the 2019-20 school year for at least one core course. 
Of those students, over 55% took English Language Arts and Mathematics, over 70% Science, and 
nearly 40% Social Science courses using Edgenuity. 

O Overall, 32.5% of all enrolled senior-grade students used Edgenuity modules at a teacher’s 
direction during the fourth grading period of the 2019-20 school year for at least one core course. 
Of those students, nearly 50% took English Language Arts, over 33% Mathematics and Science, 
and nearly 20% Social Science courses. 

e Usage. The total practice time on the modules within a given subject area is summed together for 
each student to create an aggregated usage variable. Table 4 lists for each grade, the number of users 
followed by each of three levels (Low, bottom 25%; Medium, median; and High, top 25%) of usage for 
each subject area. 


Total 


Table 4. Edgenuity Usage by Subject Area by Grade Level 


English Language Arts Mathematics 
Usage (minutes)? Usage (minutes)? 
Students (N) Low Medium — High Students (N) Low Medium High 
Middle Middle 
10,301 120 330 636 9,322 162 511 1,078 
9,177 138 316 581 9,347 164 467 937 
9,036 124 297. 552 9,663 167 465 963 
28,514 126 313 «593 28,332 164 477 991 
Senior Senior 
4,954 140 350 660 3,690 122 401 929 
4,674 149 396 §=6709 3,790 137 454 942 
3,474 126 334 601 2,135 163 479 911 
3,051 172 377 = 639 1,370 165 471 868 
16,153 145 365 658 10,985 138 446 922 
Science Social Science 
Usage (minutes)? Usage (minutes)? 
Students (N) Low Medium High Students (N) Low Medium High 
Middle Middle 
10,343 169 456 801 5,867 97 317 651 
11,057 146 360 699 8,444 135 342 630 
14,671 193 451 863 6,002 128 353 740 
36,071 171 421 791 20,313 122 339 668 
Senior Senior 
5,192 145 399 776 1,717 136 440 886 
4,406 151 385 757 108 215 467 742 
1,717 163 390 669 3,294 216 442 825 
357 268 496 774 1,118 177 387 704 
11,672 152 396 §750 6,237 192 431 816 


Note. Maximum exceeded "High" usage in English Language Arts by a factor of 12-15 times (middle grades) and 9-13 
times (senior high grades); in Mathematics by a factor of 8 times (middle grades) and 6-10 times (senior high grades); in 
Science by a factor of 7-10 times (middle grades) and 6-11 times (senior high grades); in Social Science by a factor of 9-13 
times (middle grades) and 3-8 times (senior high grades). 

*Low usage is the bottom 25%. Average usage is the midpoint. High usage is the top 25%. 


Typical usage in English Language Arts and Social Science in the middle grades was around 333 
(5.6 hours) minutes during the fourth grading period, or 42 minutes per week; and in 
Mathematics and Science was 445 minutes during the fourth grading period (56 minutes per 
week). 

Typical usage in English Language Arts and Social Science in the senior grades was around 383 
minutes during the fourth grading period (48 minutes per week) and in Mathematics and Science 
was 423 minutes during the fourth grading period (53 minutes per week). 

High usage was generally more than double those amounts. 


6. What was the effect of Edgenuity use on students’ academic performance? 


The effect of Edgenuity was examined in two ways: (a) the outcome of instruction, as measured by 
teacher-assigned academic grades in all grades 6 through 12; and (b) impact, limited to grades 6 through 
8 due to the absence of assessment data in grades 9 through 12, as measured by the relationship between 
usage and teacher-assigned grade, taking into consideration demographic variables and initial ability 
(iReady diagnostic assessment, AP2, Reading and Mathematics). Note that the impact analysis could not 
be carried out for students in senior high as no consistent measure of initial ability was available. 


English Language Arts 


e Outcomes. Table 5 lists the number of students who used Edgenuity for English Language Arts 
courses, followed by the percent who were awarded grades and the number and percent of students 
who earned academic grades of A through F during Grading Period 4. 


Table 5. English Language Arts Courses 
Academic Grade Grading Period 4 


Spring Total Percent A B C D F 
Grade Courses Graded n % n % n % n % n % 
Middle 
6 15,064 99.6 3,238 21.6 4,373 29.2 4,112 27.4 1,735 11.6 1,542 10.3 
7 13,621 99.6 3,180 23.4 3,847 28.4 3,749 27.6 1,461 10.8 1,325 9.8 
8 13,523 99.8 3,394 25.2 3,876 28.7 3,733 27.7 1,413 105 1,077 8.0 
Total 42,208 99.6 9,812 23.3 12,096 28.8 11,594 27.6 4,609 11.0 3,944 9.4 
Senior 
9 7,673 99.8 1,710 22.3 2,114 27.6 2,242 29.3 769 10.0 824 10.8 
10 7,833 99.7 1,864 23.9 2,330 29.8 2,102 26.9 729° (9.3 787 10.1 
11 5,627 99.55 1,555 27.8 1,633 29.2 1,469 26.2 492 8.8 452 8.1 
12 4,199 99.5 1,137 27.2 1,257 30.1 1,181 28.3 345 8.3 258 6.2 


Total 25,332 99.7 6,266 24.8 7,334 29.0 6,994 27.7 2,335 9.2 2,321 9.2 


Note. Counts are duplicated as students can take multiple courses. 


Oo Nearly all students received grades in the fourth grading period, of whom over 50% earned 
grades of Aor B. 

Impact. The impact of Edgenuity was analyzed by gauging whether variations in use were associated 
with variation in outcomes, once students' demographic characteristics, Prior Usage (usage of 
Edgenuity from AP2 to the shutdown), and baseline achievement were considered. The outcomes 
used were teacher-assigned grades, fit to a continuous scale of measurement where A=4 and F=0. 
Baseline achievement was measured by the iReady diagnostic assessment, AP2, administered in 
December 2019 in Reading. It should be noted that only 25 of the 46 (54.3%) possible course/grade 
units of analysis had the number of students with usage of at least 200 students, required for the 
calculation. For the purposes of the impact analysis students’ usage was divided into five bands 
(quintiles) for each course and grade. 

The expected pattern of a “dose response” for a successful program would be lower grades associated 
with low program usage, and higher grades associated with high usage. Specifically, in this analysis a 
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significant positive “dose response” is identified where one or more lower usage bands (Q1 and Q2) 


are significantly associated with lower grades and one or more higher usage bands (Q4 or QS) are 


significantly associated with higher grades as compared to that of the median (Q3). 


Table 6 lists for each English Language Arts course and grade level: the number of participants, 


median minutes of usage, and arrows depicting the direction of each usage band’s impact on the 


grading scale. Course grade combinations with a significant positive dose response, which represent 


appropriate program functioning, are shown shaded. 


O 
O 


Significant positive dose response was seen in 84% of course grade combinations. 

An increase in usage of 25 minutes per week was generally associated with an increase of around 
a one-quarter of a letter grade in the middle of the grading scale. 

Significant interaction effects were found in 6% of cases, all of which were small in magnitude 
indicating that the programs’ benefits were not limited by students’ ability. 

The incomplete dose response found in Creative Writing and Journalism may have been due to 
the relatively small number of included cases. 

The incomplete dose response found in Developmental Language through ESOL for students in 
Grades 7 and 8 suggests that higher usage was significantly associated with higher grades but 
lower usage did not have a significant effect on grades; a significant positive response was found 
in Grade 6. 


Table 6. Impact of Usage on Middle-School Teacher-Assigned Grades (Grading Period 4, 2019-20) 
English Language Arts - Course Title x Grade Level 


Usage Levels 


Median 
Usage 
Course Title Grade n(minutes) Q1 Q2 a4 Qs 
M/J Creative Writing 6 282 145 
7 258 155 -- - -- -- 
M/J Develop. Lang. Arts Thru ESOL 6 1,117 231 i 
7 776 225. - Send a 
8 880 252.0 -- 7 - oh 
M/J English Cambridge 6 1,383 438 
7 1,369 381 J ee 7 Ny 
8 1,169 438 J -- - -- 
M/J Intern. Baccalaureate Lang./Liter. 6 481 729° Of a # a 
7 556 719 
8 398 502 
M/J Intensive Reading Plus 7 225 198 
M/J Intensive Reading 6 2,652 240 
7 2,563 254 
8 2,529 241 
M/J Journalism 8 240 
M/J Language Arts Thru ESOL 6 1,098 
7 771 225 
8 884 253 
M/J Language Arts 6 3,868 244 
7 3,469 255 
8 3,300 246 
M/J Language Arts Advanced 6 3,007 354 
7 2,608 309 
8 2,945 279 


Note. Course-grade combinations with a significant positive dose response are shaded. Down arrow (J) 
indicates a usage band which is significantly associated with lower grades. Up arrow (74) indicates a usage 
band which is significantly associated with higher grades. A cell shown as dashed (“—“) indicates a usage 
band which does not have a significant effect on grades. 


Mathematics 


e Outcomes. Table 7 lists the total number of courses who attempted Mathematics courses using 
Edgenuity, followed by the percent who were awarded grades and the number and percent of 
students who earned academic grades of A through F during Grading Period 4. 


Table 7. Mathematics Courses 
Academic Grade Grading Period 4 


Spring Total Percent A B C D F 
Grade Courses Graded — n % ~ n % ~ n % ~ n % — n % 
Middle 
6 10,569 99.0 2,495 23.9 2,994 28.6 2,666 25.5 1,250 12 1,055 10.1 
7 10,726 99.8 2,621 24.5 3,028 28.3 2,689 25.1 1,255 11.7 1,113 10.4 
8 11,049 99.8 2,888 26.2 3,056 27.7 2,898 26.3 1,177 10.7 1,004 9.1 
Total 32,344 99.5 8,004 24.9 9,078 28.2 8,253 25.6 3,682 11.4 3,172 9.9 
Senior 
9 5,473 99.4 1,342 24.7 1,430 26.3 1,422 26.1 621 11.4 624 11.5 
10 5,037 99.6 1,226 24.4 1,235 24.6 1,387 27.6 589 11.7 580 11.6 
11 2,198 99.3 601 27.5 658 30.1 552 25.3 192 88 180 §=8.2 
12 1,516 99.2 345 22.9 435 28.9 478 31.8 155 10.3 91 6.1 
Total 14,224 99.4 3,514 24.8 3,758 26.6 3,839 27.1 1,557 11.0 1,475 10.4 


Note. Counts are duplicated as students can take multiple courses. 


O Nearly all students received grades in the fourth grading period, of whom over 50% earned 
grades of Aor B. 

e Impact. The impact of Edgenuity was analyzed as described under English Language Arts above. The 
iReady diagnostic assessment, AP2in Mathematics served as the measure of initial ability. It should be 
noted that only 16 of the 35 (45.7%) possible course/grade units of analysis had the number of 
students in attendance (n >= 200) as required for the calculation. In addition, 3 had incomplete data 
(fourth grading period and initial ability), so only 13 course/grade units were included in the analysis. 


Table 8 lists for each Mathematics course and grade level: the number of participants, median minutes 
of usage, and arrows depicting the direction of each usage band’s impact on the grading scale when 
compared to that of average usage. Course grade combinations with a significant positive dose 
response, which indicates that the program in functioning appropriately, are shown shaded. 


Table 8. Impact of Usage on Middle-School Teacher-Assigned Grades (Grading Period 4, 2019-20) 
Mathematics - Course Title x Grade Level 


Usage Levels 


Median 
Usage 
Course Title Grade n_ (minutes) Qi Q2 Q4 Q5 
Algebra 1 Honors 8 400 453 
M/J Mathematics 6 4,001 359 
7 4,139 349 
8 4,075 360 
M/J Mathematics Advanced 6 3,500 644 
7 2,523 566 
M/J Inter. Bacc. Mathematics 6 430 735 
M/J Intensive Mathematics 6 1,157 219 
7 1,180 212 -- -- a - 
8 1,161 206 -- - +r ‘i 
M/J Mathematics Cambridge 6 462 593 Ny; Ny = = 
7 378 1,118 NV; -- -- -- 
8 237 241 NV -- -- -- 


Note. Course-grade combinations with significant positive dose response are shaded. Down arrow (J) 
indicates a usage band which is significantly associated with lower grades. Up arrow (1\) indicates a 
usage band which is significantly associated with higher grades. A cell shown as dashed (“—“) indicates 
a usage band which does not have a significant effect on grades. 


0 Significant positive dose response was seen in 53.8% (7 of 13) of the course grade combinations. 

O Anincrease in usage of 35 minutes per week was generally associated with an increase of around 
a one-quarter of a letter grade in the middle of the grading scale. 

O Significant interaction effects were found in 16% of cases, all of which were very small in 
magnitude, indicating that the program’s benefits were not limited by the students’ initial ability. 

O In Grades 7 and 8, higher usage of Edgenuity in the Intensive Mathematics course, was 
significantly associated with higher grades, but lower usage did not have a significant impact on 
grades. 

O In Cambridge Mathematics, lower usage was significantly associated with lower grades, but 
higher usage did not have a significant impact on grades. 

Science 


e Outcomes. Table 9 lists the total number of students who attempted Science courses followed by the 


percent who were awarded grades and the number and percent of students who earned academic 
grades of A through F during Grading Period 4. 


Table 9. Science Courses 
Academic Grade Grading Period 4 


Spring Total Percent A B C D F 
Grade Courses Graded ~ n % nn % ~ n % on % on % 
Middle 
6 10,460 99.6 2,955 284 3,12830.0 2,349 22.5 977 9.4 1,011 9.7 
7 11,269 99.9 3,326 29.55 3,33529.6 2,596 23.1 913 8.1 1,087 9.7 
8 15,138 99.8 5,253 348 4,42729.3 3,205 21.2 1,206 8.0 1,023 6.8 
Total 36,867 99.8 11,534 31.4 10,89029.6 8,150 22.2 3,096 8.4 3,121 8.5 

Senior 

9 5,484 98.5 1,744 32.3 1,546 28.6 1,146 21.2 411 7.6 554 10.3 
10 4,853 99.5 1,510 31.3 1,395 28.9 1,043 21.6 439 9.1 440 9.1 
11 1,916 99.5 627 32.9 554 29.1 441 23.1 160 8.4 125 6.6 
12 398 99.5 191 48.2 91 23.0 70 17.7 20 5.1 24 «6.1 
Total 12,651 99.1 4,072 32.5 3,58628.6 2,700 21.5 1,030 8.2 1,143 9.1 


Note. Counts are duplicated as students can take multiple courses. 


O Nearly all students were graded, of whom over 60% earned grades of A or B. 

e Impact. The impact of Edgenuity was analyzed as described under Mathematics above. The iReady 
diagnostic assessment, AP2, December 2019 in both Reading Mathematics served as the measures of 
initial ability. It should be noted that only 14 of the 41 (34.1%) possible course/grade units of analysis 
had the required number of students in attendance (n >= 250) for the calculation. In addition, 3 had 
incomplete data (fourth grading period and initial ability), so only 11 course/grade units were included 
in the analysis. 


Table 10 lists for each course and grade level: the number of participants, median minutes of usage, 
and arrows depicting the direction of each usage band’s impact on the grading scale when compared 
to that of average usage. Course grade combinations with a significant positive dose response, which 
indicates that the program in functioning appropriately, are shown shaded. 
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Table 10. Impact of Usage on Middle-School Teacher-Assigned Grades (Grading Period 4, 2019-20) 
Science - Course Title x Grade Level 


Usage Levels 


Median 
Course Title Grade n Usage Qi Q2 Q4 Q5 
Inter. Bacc. M/J Comprehensive Science 6 516 671 NV NV -- -- 
7 354 458 NV; -- -- -- 
M/J Comprehensive Science 6 4,746 371 
7 4,637 252 
8 5,637 367 
M/J Comprehensive Science Advanced 6 3,325 495 
7 2,136 386 
M/J Science Cambridge 6 980 545 
7 556 398 
Physical Science Honors 7 747 514 
8 653 675 


a usage band which is significantly associated with lower grades. Up arrow (7) indicates a usage band which is 
significantly associated with higher grades. A cell shown as dashed (“—“) indicates a usage band which does not 
have a significant effect on grades. 

O Significant positive dose response was seen in only 45.5% of course grade combinations. An 
increase in usage of 35 minutes per week was generally associated with an increase of around a 
one-third of a letter grade in the middle of the grading scale. 

O Significant interactions were found in 6.8% of cases, all of which were very small in magnitude 
indicating that the program’s benefits were not influenced by the students’ initial ability. 

O In Science Cambridge, an inconsistent effect of usage was found, suggesting that usage was 
significantly associated with lower grades possibly indicating that students who practiced with 
the software did so because they needed extra help. 

O In International Baccalaureate Comprehensive Science, relative to average usage, lower usage 
was significantly associated with lower grades, but higher usage did not have a significant impact 
on grades. 


Social Science 


e Outcomes. Table 11 lists the total number of students who attempted Social Science courses followed 
by the percent who were awarded grades and the number and percent of students who earned 
academic grades of F through A during Grading Period 4. 
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Spring ‘Total 
Grade Courses 
6 5,953 
7 8,815 
8 6,166 
Total 20,934 
9 1,813 
10 132 
11 3,820 
12 1,357 
Total 7,122 


Note. Counts are duplicated as students can take multiple courses. 


Percent 
Graded 


99.9 
99.8 
99.8 
99.8 


95.4 

97 
99.5 
99.3 
98.3 


Table 11. Social Science Courses 


Academic Grade Grading Period 4 


52 
1,334 
387 
2,228 


31.8 


2,109 


30.1 


Oo Nearly all students were graded, of whom over 55% earned grades of A or B. 


e Impact. 


The impact of Edgenuity was analyzed as described under Science above. The iReady 


diagnostic assessment, AP2, December 2019 in Reading served as the measure of initial ability. It 


should be noted that only 15 of the 39 (38.5%) possible course/grade units of analysis had the required 


number of students in attendance (n >= 200) for the calculation. In addition, 2 had incomplete data 


(fourth grading period and initial ability), so only 13 course/grade units were included in the analysis. 


Table 12 lists for each course and grade level: the number of participants, median minutes of usage, 


and arrows depicting the direction of each usage band’s impact on the grading scale when compared 


to that of average usage. Course grade combinations with a significant positive dose response, which 


indicates that the program in functioning appropriately, are shown shaded. 
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Table 12. Impact of Usage on Middle-School Teacher-Assigned Grades (Grading Period 4, 2019-20) 
Social Science - Course Title x Grade Level 


Usage Levels 


Median 

Usage 
Course Title Grade n_ (minutes) Qi Q2 Q4 Q5 
M/J U.S. History 6 2,255 229 
M/J U.S. History and Career Planning 6 746 254 
M/J U.S. History Advanced 6 1,775 399 Ny NV -- -- 
Inter. Bacc. M/J U.S. History 6 225 607 -- -- -- - 
M/J Civics 7 2,275 3? RP 
M/J Civics & Career Planning 7 1,527 295 NV Ny -- -- 
M/J Civics Advanced 7 1,622 20 Para 
M/J Civics Advanced & Career Planning 7 1,374 448 NV Ny -- -- 
M/J Inter. Bacc. Civics Advanced 7 282 1,145 NV -- -- -- 
M/J World History 8 206 131 -- -- -- (hs 
M/J World History & Career Planning 8 2,976 317 aia 
M/J World History Advanced 8 202 200 . - -- - -- 


M/J World History Advanced & Career Planning 8 1,853 586 Ny Ny -- -- 


Note. Course-grade combinations with significant positive dose response are shaded. Down arrow (J) indicates a 
usage band which is significantly associated with lower grades. Up arrow (‘7\) indicates a usage band which is 
significantly associated with higher grades. A cell shown as dashed (“—“) indicates a usage band which does not have a 
significant effect on grades. 


O Significant positive dose response was seen in only 38.5% of course grade combinations. An 
increase in usage of 25 minutes per week was generally associated with an increase of around a 
one-quarter of a letter grade in the middle of the grading scale. 

O Significant interactions were found in 25% of cases, all of which were very small in magnitude 
indicating that the program’s benefits were not influenced by the students’ initial ability. 

O In most of the advanced courses, relative to average usage, lower usage was significantly 
associated with lower grades, but higher usage did not have a significant impact on grades. 

O In International Baccalaureate U.S. History and World History advanced usage did not have a 
significant effect on grades possibly due to the small number of cases available for analysis. 


7. What are the principal conclusions of this report? 


This report examined the outcomes of the Edgenuity application suites, which were widely adopted as an 
adjunct to virtual instruction during the shutdown of schools in March 2020 due to the Coronavirus 
pandemic. Edgenuity is a customizable standards-aligned video-based curriculum for middle and high 
school blended learning environments, which allows students to monitor their own progress while 
completing lessons, interactive activities, and assessments. It provides for secondary instruction and 
credit recovery in the form of a standalone (credit bearing) or a supplemental (non-credit bearing) basis. 


The sample for the analyses of participation, usage, and outcome comprised students in Grade 6 through 


12 who were enrolled in one or more core courses during the spring semester and practiced with 
Edgenuity for one minute or more during the fourth grading period. The sample for analyses of impact 
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was limited to students in Grades 6-8 having all requisite data. Impact was analyzed through a controlled 
analysis of the relationship between the length of time Edgenuity was used and the teacher-assigned 
grade. 


Edgenuity was used in 181 secondary school locations throughout the district and used under two 
different formats: as an adjunct to instruction in blended learning, and in the form of standalone courses. 
Students were given the opportunity to practice on 689 course modules across four core content areas: 
English Language Arts (n=102), Mathematics (n=284), Science (n=182), and Social Science (n=121). Over 
95% of the students who used Edgenuity, followed the blended learning format, but the percentage of 
senior high students who took standalone courses was in some cases much larger. Overall, 63.9% of all 
enrolled middle-grade students and 32.5% of senior-grade students used Edgenuity during the fourth 
quarter of the 2019-2020 school year. Typical usage in English Language Arts and Social Science was 
around 46 minutes per week and in Mathematics and Science was 55 minutes per week. Most of the 
students who used Edgenuity received teacher-assigned grades, 50% of which were A and B. 


The impact of Edgenuity for students in Grades 6 through 8 was analyzed by gauging whether variations 
in usage were associated with variations in teacher-assigned grades once students' demographic 
characteristics, prior usage, and baseline achievement on the iReady diagnostic assessment. The program 
was considered to have been effective if at least one band of below average usage was significantly 
associated with lower grades and at least one band of higher usage was significantly associated with 
higher grades once all other factors were considered. 


Significant positive dose response was seen for middle-grade students in 84% of the English Language Arts 
courses/grade combinations, 53.8% of those in Mathematics, 45.5% in Science, and 38.5% in Social 
Science. An increase in usage of 25-35 minutes per week was generally associated with an increase of 
around a one-quarter to one-third of a letter grade in the middle of the grading scale. Further analysis 
indicated the program’s benefits tended to be equally effective across all initial ability levels. The middle 
school students in the Cambridge and International Baccalaureate courses were least likely to have been 
impacted by completing Edgenuity modules, and those in the standard education and intensive courses 
were most consistently positively impacted. 
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